PROPER HOLOMORPHIC MAPPINGS 
OF THE SPECTRAL UNIT BALL 
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Abstract. We prove an Alexander type theorem for the spectral unit ball Q n show- 
ing that there are no non-trivial proper holomorphic mappings in Q n , n > 2. 



Let M n denote the space of n x n complex matrices. 

In order to avoid some trivialities and ambiguities we assume in the whole paper 
that n > 2. 

Let p(A) := max{|A| : A G Spec(A)} be the spectral radius of A G M n - Denote 
also by Spec(A) :— {A G C : det(/l — AI„) = 0} the spectrum of A G M n , where 
the eigenvalues are counted with multiplicities (I„ denotes the identity matrix). 
We also denote the spectral unit ball by f2 n := {A G M n : p(A) < 1}. Note that 

2 

fl n is an unbounded pscudoconvex balanced domain in C with the continuous 
Minkowski functional equal to p. For A G M n denote -Pa (A) := dct(AI„ — A) = 
A" + £" =1 (-l) J Vj(.A)A n_J ', A G M n . Denote also a := {a u ... ,a n ). We put 
G n '■= o~(Q n ). The domain G n is called the symmetrized poly disc. Note that 
cr G 0(M n , G„). Denote also J n := 7T„({(Ci, . . . , ( n ) : ( 3 = ( k for some j ^ k}), 
where 7r n)j (Ci,... ,( n ) := Ei<fc 1 <...<fc J <„ Cfcj '•••'(%) G B, i = 1, . . . , n. (B 
denotes the unit disc in C). Note that G n \ J n is a domain and G„ \ »7n is dense in 
G„. 

Note that Q n = U 2G g„ T ^ where T z := {A G fi» : cr(A) = z}, z G C". The sets 
T z , z G C" are pairwise disjoint analytic sets. Note that if the matrix A e T z is non- 
degoratory then A is a regular point of T z - recall that in such a case rank cr'(A) = n 
- it is the largest possible number. For definition and basic properties of non- 
derogatory matrices see [Nik-Tho-Zwo 2007] and references there. One of possible 
definitions of a non-derogatory matrix is that different blocks in the Jordan normal 
form correspond to different eigenvalues (or equivalently all eigenspaces are one- 
dimensional). We shall deliver some properties of the sets T z (see Lemma 5, Lemma 
6 and Corollary 7). It is also simple to see that Tq is a cone which contains at least 
n 2 — n + 1 linearly independent vectors: for instance the ones consisting of one 
1 lying not on the diagonal (and with other entries equal to 0) and the matrix 
A = (Oj,fc)j,fc=i,... ,n such that a\ y \ = 1, a\.2 = —1, 02.1 = 1, £12,2 = —1 (and with all 
other entries equal to 0). Consequently, we shall see that is not a regular point 
of Tq. On the other hand the sets T 7rn (( 1 .... ) f„), where the points Ci? ■ • ■ iCn G B are 
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pairwise different, are submanifolds - it follows from the fact that in this case all 
elements of %v n (Ci,— Xn) are non-derogatory. 

It is well-known that for a given mapping F G 0(fi n , fin) there exists a mapping 
F G 0(G„,G„) such that cr(F(A)) = F(a(A)) (see e.g. [Edi-Zwo 2005]). 

If / G 0(B,D) then one may well-define the following holomorphic mapping 

F f : Q n 3 A ~ f(A) := ^r 1 ^ G fi„. Note that F/(a(A)) = <r(F/(A)) = 

7i"n(/(Ai), • • • , /(A„)), where cr(A) = 7r n (Ai, . . . , A„). In particular, F a G Aut(fi„) 
for any a G Aut D. On the other hand the function Fg, where B(X) — A 2 , A G ID is 
a mapping of the form £1 2 3 A A 2 G 2 , which is not a proper holomorphic one 
- it maps T into 0. 

The structure of the group of automorphisms of Q n has been been studied in 
several papers (see e.g. [Ran-Whi 1991] and [Ros 2003]). However, it is still not 
understood completely. Let us mention only that Aut(f2„) is not transitive. Moti- 
vated by the results of the mentioned papers we are going to examine the structure 
of the class of proper holomorphic self-mappings of the spectral unit ball. It turns 
out that we get an analogue of the theorem of Alexander on proper holomorphic self 
mappings of the Euclidean ball in C™ stating that there are no non-trivial proper 
holomorphic self maps in the unit ball B„, n > 2 (see [Ale 1977]). 

In the paper we need some properties of proper holomorphic mappings between 
complex analytic sets that could be found in [Chi 1989] and [Loj 1991]. The book 
[Rud 1980] may serve as another reference on proper holomorphic mappings (mostly 
between open sets in C"). 

Theorem 1. Let F : Q n Sl„ be a proper holomorphic mapping, n > 2. Then F 
is an automorphism. 

The following necessary form of proper holomorphic mappings of the spectral 
ball, which is a simple consequence of the description of the set of proper holomor- 
phic self-mappings of the symmetrized polydisc, will be crucial in our considerations 
and justifies the introducing of the condition (1) below. 

Proposition 2 (see Theorem 17 in [Edi-Zwo 2005]). Let F : Q, n i— > J7„ be a proper 
holomorphic mapping. Then there is a non- constant finite Blaschke product B such 
that cr{F(A)) = n n (B(Ci), ■ ■ ■ ,B(( n )), where A E Q n and a(A) = tt„(Ci, . . . , ( n ), 
,Cn GD. 

In view of Proposition 2 it is natural that we study below the holomorphic 
mappings F : Q n i— > Sl„ such that there is a function / G 0(ID, B) with the property 

(1) a(F(A)) = 7r„(/(Ci), . . . ,/(Cn)), A G fi n is such that a{A) =*„(&,... ,C«)- 

We start with the following lemma. 

Lemma 3. Let F G £>(£!„, 0„) be such that F(0) = and (1) is satisfied for 
f G 0(E>, D) (then necessarily /(0) = 0) with f (0) ^ 0. Then F'(0) is a linear 
isomorphism (of A4 n ). 

Proof. Put a := /'(0). Fix V G M n . Let 7r„(/z) = a(V) for some \i = (fii, . . . , [i n ) G 
C". We first prove that 



(2) 



-F'(0)(V)ea(V). 
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Actually, <j((V) — 7r„((ji), ( E C Consequently, 

<7(F(CV))=7T„(/(CMl),-- - ,/(0*n)) 

for sufficiently small £ £ B and then 

/ (/((ft),... ,/(CMn)) \ 

CT l~J =7r " l c J' 

Passing with ( to we get that F'(0)(V) G % n{ail) . Therefore, 4> := ^F'(O) : 
.M„ i ^ Al„ is a linear mapping such that 

(3) $(T z )cT 2 , zeC". 

To finish the proof of the lemma it is sufficient to show that $ is a monomorphism. 
Suppose that it does not hold. Then there is an N G M. n , N ^ such that 
$(JV) = 0. Because of (3) we get that N E%. But then there is an M G % such 
that N + M In particular, 

T J <f>(N + M) = <f>(N) + $(M) = $(M) e T 

- contradiction. 
□ 

Lemma 4. Let i* 1 G (9(fi n ,fi n ) 6e st/cft iftai (1) is satisfied with /(0) = and 
f (0) ^ 0. Tften F _1 (0) n T C {0}. 

Proof. Suppose that there is an A G T , A ^ such that = 0. It follows 

from the Jordan decomposition theorem that there are linearly independent vectors 
Vi,V2 G A4 n such that A(v2) — t>i, A(v\) — (at the moment it is essential that 
n > 2). Let (vi, . . . , v n ) be a vector base of C™. Define the linear mapping V : C n 
C™ (equivalently an element from M n ) as follows ^(^2) := v\, V{v\) := v 2 and 
V(vj) := 0, j = 3, . . . ,n. Then (A + C^) 2 (uj) = C(l + C)«j, 3 = 1,2. Consequently, 
the properties of the spectral radius imply that 

m+c\<p{{A+cvf)< P 2 {A + cv). 

For any ( G C there are /Uj(C) G C, j = 1, . . . , n such that 7r„(^i(C), ■ • ■ , Mn(0) — 
<r(A+"cn Then <t(F(A + <V)) - ^(/(Mi(0), • • • ,/(MC))) for ( gM sm all. 
We also know that max{|^(C)| : j = 1, . . . , n} = p(A + (V) > \/\C\\/\l + (\ and 
p{A + (V) -> as C 0. 

Note that p( F(A + cy) ) -► p(F'(A)0O) as C -» 0. But on the other hand 

( F(A + (V) \ [ 1/(^(0)1 . , \ 
P l C J = — { ^ = 1,--- 

which tends to infinity as ( — * because /'(0) 7^ - a contradiction. □ 

Note that the results proven so far referred to a larger class of mappings than 
only proper ones. It is possible that they may have application to the study of more 
general mappings than only the proper holomorphic ones. 

First we show simple results on the geometry of the sets T z . 
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Lemma 5. The set of non- derogatory matrices is dense in T z for any z G G„. 

Proof. Fix z G G„. Let 4 e T z . Without loss of generality assume that A is not 
non-derogatory. Choose a vector base B in which A has Jordan normal form. Let 
us study two different blocks corresponding to the same A (and the corresponding 
vectors from B: v\, . . . , Vk, w\, . . . ,wi, k,l > 1). Let Av\ = Xv\, Avj = Xvj + Vj-\ , 
j = 2, ... ,k, Aw i = Xwi, Awj = Xwj + Wj-i, j = 1, ... ,Z. For e > define 
Bv\ := Xvi + ewi and for all other elements of the base B define Bv :— Av, v G B, 
v ^ Vi . This easily gives an approximation of A with matrices still in T z having one 
block corresponding to the eigenvalue A less than in the original matrix. Repeating 
this procedure for all Jordan blocks having the same eigenvalues we easily construct 
a sequence of non-derogatory matrices in T z tending to A. □ 

Lemma 6. The set of non- derogatory matrices in T z is connected and open in T z 
for any z G G n . 

Proof. Fix z G G„. The non-trivial part of the lemma is the connectedness. Let 
us fix a system of numbers (Ci, ■ • ■ , Cn) an d the sequence of indices 1 = k\ < ki < 
... < ki + i = n + 1 where ( kj = ( k] +i = ... = Ck j+1 -i, j = 1,... ,1 (and such that 
no other equalities between different Q's hold) and 7r„(Ci, • • • , Cn) = z - And now 
for any vector base (v\, . . . ,v n ) of C n we define the matrix A (more precisely, an 
element in T z C M n ) as follows Av t = (jV t + Vt-i, j = 1, • • • , I, kj + 1 < t < fcj+i, 
Av kj — (jVkj, j = 1, . . . , I. Note that the above mapping is continuous and its 
image equals the set of non-derogatory matrices in ^ r „(f 1 ,... ,£„)■ This together with 
the fact that the set of all vector basis is connected in A4 n completes the proof. □ 

As a simple corollary of the results on the set of non-derogatory matrices in the 
sets T z we get the following. 

Corollary 7. For any z G G„ the set T z is an analytic irreducible set of codimen- 
sion n. 

At the moment we are ready to move to the proof of our main result. 

Proof of Theorem 1. First recall that when F : 0„ i— > Cl n is a proper holomor- 
phic mapping then there is a finite non-constant Blaschkc product B such that 
o-(F(A)) = TT n (B(d), . . . ,B(( n )), where a(A) = tt„(Ci,... ,(n)- In particular, 
^(^r„(Ci,-,C«)) c ^n(s(Ci),-,B(Cn))i G D, j = l,...,n- But the proper- 
ness of F implies even that the equality -F 1 (^ r „(Ci.... ,c re )) = %r n (B(d),... .s(C„)) i 
Q G D, holds - it is sufficient to note that T z is always connected. Even more, 

F \ T * n (Ci,- ,Cn) : T -*n«i,- Xn) ^ T *n (b(Ci ) , • • ■ MM) is °P CI1 and proper for any Q G D, 
j = 1, . . . ,n. 

We claim that for any A G D such that B'(Xq) ^ (note that such points exist) 
the function 

( 4 ) F I^„(a A 0) is injective. 

Actually, making use of the automorphisms of fl n and the properties of Blaschkc 
products we may assume that A = 0, B(0) = and B'(0) ^ 0. It follows from 
Lemma 4 that F^ 1 ^) D % — {0}. In particular, F(0) = 0. Now Lemma 3 applies 
and we get that F'(0) is an isomorphism. Consequently, F is locally invertible near 
0. Note that there is a neighborhood V of such that #F _1 (^4) n % = 1 for any 
A G VflT . Otherwise there would exist T 3 A" ,A V such that A v ^ A v and 
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F(A V ) = F(A V ) — > 0. But the properness of F implies that (taking if necessary a 
subsequence) either both sequences (A"), (A v ) converge to or at least one of the 
sequences converges to a non-zero element A from Ti such that F{A) = 0. In the 
first case we contradict the local invertibility of F near and in the second case we 
get two points in .F _1 (0) fl 7o - a contradiction, too. 

Now the analyticity of the set {A G T : n% = 1} (see e.g. [Loj 1991], 

Section V.7.1) (the mapping F\ To : % % is proper and open) and the fact that 
7o is a cone shows that the mapping Fif : To i— > Tq is a one-to-one mapping. 

Now we prove the following property. 

(5) Let z v — > z° G G„, where z v G G„ be such that Fyr z „ is not injective for any v 
then Fyr is not injective. 

Actually, to prove (5) note that because of the properties of proper holomorphic 
mappings we may assume that there are two sequences of non-derogatory matrices 
(A u ), (A v ) with A v ^ A" lying in 7>, F(A V ) = F(A U ) and tending to matrices 
A,A<E T z o such that A is non-derogatory and Fyr is locally invertiblc in A. In 
the case A / Awe are done, so assume that A = A. The local invertibility of F\j 
near A implies that there is a neighborhood U of A such that Fyr^riU is invertiblc 
for v large enough, which contradicts the equality F(A V ) = F(A V ). 

We claim that 

(6) for any z G G„ the mapping Fij- z is injective. 

Put U := {z G G„ : Fi Tz is injective}. The fact that F\ To is injective shows 
that U is not empty. The property (5) shows that U is open. To see that U is 
closed in G n take a sequence U 3 z v — > z G G n . Suppose that z ^ U. Then 
there arc different non-derogatory matrices A\,.. . , Ak G T z , C G T w with k > 2 
such that i^ _1 (C) PI T z = {A\, . . . ,Ak}. We may choose arbitrarily small open 
connected neighborhoods U\, . . . ,Uk, V of A\, . . . ,Ak,C such that Ui C\U P = for 
I ^ p, Uj n T z is connected, V n 7^ is connected for any z, w G G n , j — 1, . . . , k 
and F _1 (V) C Uj=i^j- Consequently, for any ^ there arc pairwisc disjoint sets 
F(lij (~1 T z u) (~l V, j = 1, . . . , k that are open in V fl T w v and that are non-empty for 
v large enough. Now the properness of F shows that for V sufficiently small the 
sets F(Uj fl T z v), j — 1, . . . , k cover the whole set V fl T w v for v large enough; thus 
contradicting the connectedness of V n T w v. 

Since G„ is connected we get that U = G„, so (6) is satisfied. 

Let to denote the degree of B. We claim that m = 1. Suppose that to > 2. Note 
that taking instead of F the composition of many F's we may assume that to > n. 
There is a point Co G ^ such that _B~ 1 (Co) = {Cii • ■ ■ > Cm}- Composing, if necessary, 
with automorphisms of 0„ we may assume that Co = 0. Recall that ^r n (Ci,... ,f n ) is 
an n 2 — n dimensional submanifold. Choose A G %r n (Cu—M sucn ^ na * ^(^) = 0- 

Let / := F\r 7 , riiCi <n) '■ T Wn ^ 1 ^ %. Then / is a holomorphic bijective 

mapping. Let us fix a regular point C in %. Then the function 

^CsAH/^eT^ w 

is holomorphic on C \ {0} (the points AC, A G C \ {0}, are regular in T ) and 
continuous at with ^>(0) = A (use the properness and injectivity of /). Con- 
sequently, ip is holomorphic on C. Note that (F o <p)(A) = AC, A G C, so the 

tangent space to ^r n (Ci,... ,f n ) at A i.e. T^Tw^ f n )) is mapped onto i? := 

^'(^(^(r^^^...^"))),' which contains the vector F' (a) (<p'(0)) = (Foip)'(0) = C. 
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Consequently, H contains all regular points of Tq, so it contains the whole 7$, which 
contains n 2 — n + 1 linearly independent vectors contradicting the fact that H is at 
most n 2 — n dimensional vector space. 

Consequently, we have proven that #F _1 (C) = 1 for C G 0„ showing that F is 
an automorphism. □ 

Acknowledgment. The author wishes to express his gratitude to Witold Jarnicki 
for fruitful conversations on the properties of proper holomorphic mappings between 
analytic sets. 
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